In high dimensional settings, density estimation algorithms rely crucially on their inductive bias. Despite recent empirical success, the inductive bias of deep generative models is not well understood. In this paper we propose a framework to systematically investigate bias and generalization in deep generative models of images by probing the learning algorithm with carefully designed training datasets. By measuring properties of the learned distribution, we are able to find interesting patterns of generalization. We verify that these patterns are consistent across datasets, common models and architectures.

After reading author responses and discussing with other reviewers, I have decided to raise my score. I think the authors did a good job in their response to the points I raised. However, I still think that their should be more emphasis in the paper on the significance of the observations made in the paper which was not clear to me at first. The study relies on probative experiments using synthetic image datasets (e.g. CLEVR, colored dots, pie shapes with various color proportions) in which observations can be explained by few, independent factors or features (e.g.

In high dimensional settings, density estimation algorithms rely crucially on their inductive bias. Despite recent empirical success, the inductive bias of deep generative models is not well understood. In this paper we propose a framework to systematically investigate bias and generalization in deep generative models of images by probing the learning algorithm with carefully designed training datasets. By measuring properties of the learned distribution, we are able to find interesting patterns of generalization. We verify that these patterns are consistent across datasets, common models and architectures.